Clean up stale `augment-vis` saved objs #4059

ohltyler · 2023-05-17T22:27:44Z

Description

This PR adds cleanup of stale/outdated/dangling augment-vis saved objs.

These objs can become "stale" if either (1) the associated (referenced) visualization has been deleted, or (2) the associated plugin resource has been deleted.

Cleaning up after deleted plugin resources

In the render workflow of a visualization, when fetching all of it's augment-vis saved objs, if any include a deleted plugin resource, delete it. See the new helper fn cleanupStaleObjects(). DEMO: see Figure 1 below.

Cleaning up after deleted visualizations

We update the vis saved object loader to implement a custom delete() fn which will fetch all augment-vis saved objs that have this vis id and delete it. To make sure this loader function is called, we update a few places where deletion of a visualization may commonly occur:

Visualize plugin - table page. DEMO: see Figure 2 below.
Saved objects management plugin: In the 'Object View' page (when inspecting a saved object of visualization type). DEMO: see Figure 3 below.
~~Saved objects management plugin: table page~~ TODO: this may not be done as part of this effort. A lot of edge cases and batch processing scenarios we need to consider. Is also more of an edge case

Additional info

This also adds a RESOURCE_DELETED error type such that plugins can propagate this information to indicate their resource has been deleted. This is required in order to be picked up in cleanupStaleObjects() to get cleaned up.
Plugins may also add logic of their own to clean up any dangling objects themselves at the time of plugin resource deletion. The purpose of adding deletion logic here is to take the burden off of plugins, and let cleanup happen automatically upon each fetch.

Demo videos

Figure 1: Given 2 stale augment-vis saved objects with deleted plugin resources, when re-rendering the visualization that is associated 2 these 2 objects, they are cleaned up and deleted in the background. Confirmed via .kibana index
stale-plugin-resource-demo.webm

Figure 2: Given a visualization with 1 augment-vis saved object, when deleting the visualization from visualize plugin, the augment-vis saved object is cleaned up in the background. Confirmed via .kibana index
stale-vis-visualize-plugin-demo.webm

Figure 3: Given a visualization with 1 augment-vis saved object, when deleting the visualization from saved objects management plugin's edit object page, the augment-vis saved object is cleaned up in the background. Confirmed via .kibana index
stale-vis-saved-obj-edit-demo.webm

Issues Resolved

Closes #4044

Testing the changes

Check List

src/plugins/visualize/public/application/components/visualize_listing.tsx

ohltyler · 2023-05-26T00:48:03Z

Thanks for the detailed proposal @ashwin-pc . Agreed this is a better strategy and removes the side-effect way of using the augment vis loader within the visualization loader's delete(). I've finished making those changes you suggested - I had to take a dependency on uiActions in a few places, but I figure that's for the best anyways so it can be used in the future.

I also wonder if uiActions belongs in core, because it seems to look that way...

joshuarrrr

Looks mostly good - thanks @ashwin-pc for the suggestion, I like this approach, and I agree that we may want to make UIActions even more accessible via core in the future.

I do want to verify the ability to delete non-visualization objects like visbuilder. Other than that, the only other blocker is the commented out sections.

src/plugins/visualize/public/application/components/visualize_listing.tsx

src/plugins/saved_objects_management/public/triggers/saved_object_delete_trigger.ts

joshuarrrr · 2023-05-26T00:50:00Z

src/plugins/saved_objects_management/public/services/service_registry.ts

@@ -32,7 +32,7 @@ import { SavedObjectLoader } from '../../../saved_objects/public';

 export interface SavedObjectsManagementServiceRegistryEntry {
  id: string;
-  service: SavedObjectLoader;
+  service: SavedObjectLoader | any;


does unknown also work here?

src/plugins/vis_augmenter/public/ui_actions_bootstrap.ts

src/plugins/vis_augmenter/public/actions/plugin_resource_delete_action.ts

src/plugins/vis_augmenter/public/actions/saved_object_delete_action.ts

joshuarrrr · 2023-05-26T05:52:58Z

@ohltyler There's also one failing unit test:

 FAIL  src/plugins/saved_objects_management/public/plugin.test.ts
  ● SavedObjectsManagementPlugin › #setup › registers the saved_objects feature to the home plugin

    TypeError: Cannot read property 'registerTrigger' of undefined

      19 |
      20 | export const bootstrap = (uiActions: UiActionsSetup) => {
    > 21 |   uiActions.registerTrigger(savedObjectDeleteTrigger);
         |             ^
      22 | };
      23 |

      at bootstrap (src/plugins/saved_objects_management/public/ui_actions_bootstrap.ts:21:13)
      at SavedObjectsManagementPlugin.setup (src/plugins/saved_objects_management/public/plugin.ts:145:5)
      at Object.<anonymous> (src/plugins/saved_objects_management/public/plugin.test.ts:52:20)

ohltyler · 2023-05-26T16:13:12Z

@joshuarrrr addressed comments, fixed test.

...plugins/saved_objects_management/public/management_section/object_view/saved_object_view.tsx

src/plugins/saved_objects_management/public/register_services.ts

src/plugins/saved_objects_management/public/plugin.ts

ashwin-pc · 2023-05-26T17:43:42Z

src/plugins/saved_objects_management/public/services/service_registry.ts

@@ -32,7 +32,7 @@ import { SavedObjectLoader } from '../../../saved_objects/public';

 export interface SavedObjectsManagementServiceRegistryEntry {
  id: string;
-  service: SavedObjectLoader;


Same as above, Why do we need this?

See above comment

src/plugins/vis_augmenter/public/actions/saved_object_delete_action.ts

src/plugins/visualize/public/application/components/visualize_listing.tsx

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

src/plugins/saved_objects_management/public/register_services.ts

src/plugins/saved_objects_management/public/plugin.ts

...plugins/saved_objects_management/public/management_section/object_view/saved_object_view.tsx

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

ashwin-pc

Perfect! thanks for making those changes :)

ohltyler requested review from ananzh, kavilla, seanneumann, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, kristenTian, zhongnansu and manasvinibs as code owners May 17, 2023 22:27

ohltyler added the v2.8.0 label May 17, 2023

ohltyler marked this pull request as draft May 17, 2023 22:27

github-actions bot added the valued-contributor label May 17, 2023

ohltyler added visualizations Issues and PRs related to visualizations unified visualization UX feature-anywhere and removed valued-contributor labels May 17, 2023

ohltyler removed request for seanneumann, joshuarrrr, AMoo-Miki, kavilla, ashwin-pc, zhongnansu, abbyhu2000, zengyan-amazon, ananzh, kristenTian and manasvinibs May 17, 2023 22:28

ohltyler mentioned this pull request May 17, 2023

[Feature Anywhere] Enable plugins to augment visualizations with additional data and context #2880

Closed

31 tasks

ohltyler commented May 26, 2023

View reviewed changes

src/plugins/visualize/public/application/components/visualize_listing.tsx Outdated Show resolved Hide resolved

joshuarrrr reviewed May 26, 2023

View reviewed changes

ashwin-pc reviewed May 26, 2023

View reviewed changes

ohltyler added 12 commits May 30, 2023 17:39

Clean up stale augment-vis saved objs

630cfaf

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

minor cleanup

c94a8a8

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

More cleanup

d5c7d0e

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Add triggers/actions in visaugmenter

3c43148

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Add trigger and logic to use trigger in saved obj management

5930f9b

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Confirm all workflows are working

304bb54

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Clean up

a913317

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Minor comments & vis delete bug fix

d393090

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Remove comments in bootstrap file

0e89335

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Add tests for saved_object_delete_action

1064e21

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

update visualize_listing; add test for other action;

62fec5b

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Cleanup after rebasing

67ae9ab

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

ohltyler force-pushed the saved-obj-cleanup branch from 34c2a21 to 67ae9ab Compare May 31, 2023 01:06

More cleanup

c8d2456

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

ashwin-pc reviewed May 31, 2023

View reviewed changes

ohltyler added 2 commits May 31, 2023 08:40

Update tests after rebasing

e7fbedc

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

Remove UIActions from start services

e709989

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

ashwin-pc approved these changes May 31, 2023

View reviewed changes

ohltyler merged commit 538d91e into opensearch-project:feature/feature-anywhere May 31, 2023

ohltyler deleted the saved-obj-cleanup branch May 31, 2023 17:04

ohltyler mentioned this pull request May 31, 2023

[Feature Anywhere] Cleanup of dangling augment-vis saved objs #4044

Closed

kavilla added v2.9.0 and removed v2.8.0 labels Jun 1, 2023

ohltyler mentioned this pull request Jul 6, 2023

Introduce automatic cleanup of related saved objects #4499

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up stale `augment-vis` saved objs #4059

Clean up stale `augment-vis` saved objs #4059

ohltyler commented May 17, 2023 •

edited

Loading

ohltyler commented May 26, 2023 •

edited

Loading

joshuarrrr left a comment

joshuarrrr May 26, 2023

joshuarrrr commented May 26, 2023

ohltyler commented May 26, 2023

ashwin-pc May 26, 2023

ohltyler May 26, 2023

ohltyler May 31, 2023

ashwin-pc left a comment

Clean up stale augment-vis saved objs #4059

Clean up stale augment-vis saved objs #4059

Conversation

ohltyler commented May 17, 2023 • edited Loading

Description

Cleaning up after deleted plugin resources

Cleaning up after deleted visualizations

Additional info

Demo videos

Issues Resolved

Testing the changes

Check List

ohltyler commented May 26, 2023 • edited Loading

joshuarrrr left a comment

Choose a reason for hiding this comment

joshuarrrr May 26, 2023

Choose a reason for hiding this comment

joshuarrrr commented May 26, 2023

ohltyler commented May 26, 2023

ashwin-pc May 26, 2023

Choose a reason for hiding this comment

ohltyler May 26, 2023

Choose a reason for hiding this comment

ohltyler May 31, 2023

Choose a reason for hiding this comment

ashwin-pc left a comment

Choose a reason for hiding this comment

Clean up stale `augment-vis` saved objs #4059

Clean up stale `augment-vis` saved objs #4059

ohltyler commented May 17, 2023 •

edited

Loading

ohltyler commented May 26, 2023 •

edited

Loading